Reading Diary 3

5 Predicting Terms in Hypo-hypernym Relations with Pre-trained Transformers. terms

bag-of-word

It is a particular representation model used to simplify the contents of a selection of text.

Example: The MLP model simply takes bag-of-word embeddings to make dialogue act prediction, and the RNN model is a bi-directional GRU network [3].

fine-tuning

The procedure of re-training a pre-trained language model using your own custom data.

Example: This seems to be consistent with the observation by Tayyar Madabushi et al. (2021) that fine-tuning is indeed a reasonable way of learning the representation of MWEs [2].

lexicon

It is a part of the grammar of a language that includes all lexical entities.

Example: Moreover, MWEs are pervasive in all domains (Biber et al., 1999), and it has been estimated that their size in a speaker’s lexicon of any language is of the same order of magnitude as the number of single words [2].

n-grams

Diving the corpus into n-words chunks.

Example: Second, only terms up to trigrams were considered [1].

parts-of-speech (POS)

It consists of assigning a category tag to the tokenized parts of a sentence.

Example: In addition, since MWEs that are named entities are usually literal, we use a part-of-speech tagger to identify proper nouns [2].